Internet Archive's in-browser bookreader "theater" requires JavaScript to be enabled. It appears your browser does not have it turned on. Please see your browser settings for this feature.

Big Data for Beginners

by: Huybrechts, Pieter

Publication date: 2023-8-18
Usage: Attribution 4.0 International
Topics: data integration, biodiversity data
Publisher: Pensoft Publishers
Collection: biodiversity
Contributor: Pensoft Publishers
Language: English
Rights: https://biodiversitylibrary.org/permissions
Rights-holder: Copyright held by individual article author(s).
Volume: 7
Item Size: 1.7M

Abstract: With the increasing amount of datasets being published and made available through global aggregators, such as the Global Biodiversity Information Facility (GBIF), new opportunities have opened to answer research questions that previously could not be considered. Techniques for large scale data integration offer benefits for the biodiversity research community (Heberling et al. 2021, Kays et al. 2020), profiting from the great and continuing efforts in data mobilisation and standardisation (such as Darwin Core, Wieczorek et al. 2012). These benefits include integrating several large data sources or enriching existing occurrence data with other information. Several commonly encountered barriers to large-scale use of biodiversity occurrence data exist. These include the lack of facilities for local storage of large and rapidly changing datasets, the computational power required for processing, unfamiliarity with existing toolsets, and insufficient resources to maintain big data infrastructure. These challenges are well documented in the context of high-throughput genomics (Marx 2013), and more recently in occurrence-based biodiversity research (for example Thessen et al. 2018).
However, while these hurdles and bottlenecks are very real, several of them have low cost of entry solutions. The aim of this presentation is to encourage the community to explore ambitious queries, to combine and examine all available data in its totality and to break down specific technical barriers, by providing a practical overview for researchers to maximise the power of large-scale data processing in their work.
While big data processing may seem daunting, tools accessible to users without a background in big data are available for both local workstations and cloud computing services that allow for scalable data processing at low cost, for instance Databricks Community Edition or Apache Arrow. Using these resources, researchers can incorporate larger datasets into existing protocols, and by doing so, uncover patterns and insights that would be otherwise impossible to acquire using smaller subsets of the ever-expanding complex set that biodiversity occurrence data presents.
Addeddate: 2025-06-09 20:16:34
Bhl_virtual_titleid: 210882
Bhl_virtual_volume: v.7 (2023)
Call number: 10_3897_biss_7_111301
Call-number: 10_3897_biss_7_111301
Foldoutcount: 0
Genre: article
Identifier: bigdatabeginner7huyb
Identifier-ark: ark:/13960/s2fm80zcdws
Identifier-bib: 10_3897_biss_7_111301
Identifier-doi: 10.3897/biss.7.111301
Ocr: tesseract 5.3.0-6-g76ae
Ocr_detected_lang: en
Ocr_detected_lang_conf: 1.0000
Ocr_detected_script: Latin
Ocr_detected_script_conf: 1.0000
Ocr_module_version: 0.0.21
Ocr_parameters: -l eng
Page_number_confidence: 0
Page_number_module_version: 1.0.5
Page_range: 1-2
Pages: 2
Pdf_degraded: invalid-jp2-headers
Pdf_module_version: 0.0.25
Possible copyright status: In copyright. Digitized with the permission of the rights holder.
Ppi: 300
Source: Biodiversity Information Science and Standards 7
Year: 2023